feat(refid): generate agency reference IDs on application inject - #307

Description

An agency had no way to issue its own reference number for an application — the only identifier was the opaque NSW task ID. Where a number was needed it was typed by hand into the review form, with nothing guaranteeing it unique, sequential, correctly formatted, or scoped to the issuing office.

This adopts core/refid, split across two config layers so that what an agency can issue is a deployment decision while which tasks get one is a task decision:

• refIDGen in config.yaml declares the formats. Optional — omit it and no task can generate a reference ID.
• A new optional refid block in a task config names an (issuer, idType) from there, the JSON Pointer to store the result at, and params mapped to JSON Pointers into the injected data — which is what lets one task config serve every office rather than needing one per office.

Generated once, on first inject; a re-inject keeps the number it already has. Generation failure fails the inject, so an application never exists without its reference ID.

Its dependency, OpenNSW/core#186 (stop registering database/sql drivers in store subpackages), is merged and backend/go.mod now points at it — no replace directive remains. Without that fix this binary panics at startup with sql: Register called twice for driver sqlite, since refid/store/sqlite and the GORM sqlite driver both registered the name.

Type of Change

• New feature (non-breaking change which adds functionality)

Changes Made

• internal/taskconfig — optional TaskConfig.RefID, validated like the existing consignmentFields pointers. CurrentSchemaVersion stays at 1.
• internal/application — refid.go resolves params and writes the ID at path; service.go generates only for a new application, immediately before the store write. Also carries ReviewerResponse forward on re-inject, which CreateOrUpdate's full-row Save would otherwise NULL out, destroying an issued ID.
• internal/refidstore — selects the upstream backend by dialect, reusing the existing GORM pool (a second sql.Open is a different database for sqlite :memory:).
• migrations/000010 — the counter table, via this repo's migrator so the .sql file stays the source of truth and it gets down/status.
• cmd/server — finishes the refIDGen plumbing (it was decoded and discarded) and builds the registry, which validates every format at boot.
• Docs + agency-neutral commented examples in config.example.yaml and values-example.yaml.

Errors: an unresolvable param or a value outside a configured list → 400; an unconfigured issuer/idType, counter overflow, or a DB error → 500.

Testing Details

Test Environment: local — SQLite via start-dev.sh, plus Postgres for the migration check.

go build ./... && go vet ./... && go test ./... — 26 packages pass, gofmt/vet clean. 19 new tests, notably:

• internal/refidstore drives the upstream sqlite store through this module's real driver (glebarez, not modernc). refid's queries use RETURNING and ?N placeholders that upstream only tests against modernc, so this is the compatibility proof.
• internal/application covers generation, re-inject preserving the ID, an unconfigured registry failing the inject with no row created, and an end-to-end test with the real registry and counter table (same office → 000001/000002, another office → 000001, unlisted office → rejected).

Also verified by hand: migrate up/status/down on both SQLite and Postgres; the server boots logging reference ID generation configured issuers=1 (the regression check for the driver panic the core PR fixes); helm template renders refIDGen into both ConfigMaps with {issuer} and {{env:...}} intact; and removing the ReviewerResponse carry-forward makes the re-inject test fail, confirming it catches the data loss.

Manual test guide (NPQS)

1. Add to backend/config/npqs/config.yaml:

refIDGen:
  issuers:
    - issuer: NPQS
      formats:
        # NPQS/NPQS-KAT/20260904/000001 — per office, reset daily.
        - idType: application_id
          segments:
            - { type: literal, value: "NPQS/" }
            - { type: list, list: office_location, param: officeCode }
            - { type: literal, value: "/" }
            - { type: date, layout: "20060102" }
            - { type: literal, value: "/" }
            - { type: sequence, padding: 6,
                scopeKey: "{issuer}:{idType}:{officeCode}:{yyyyMMdd}" }
  lists:
    # Must match nppo_office_location's codes in one-trade-artifacts exactly —
    # a list segment rejects anything else and the inject then fails closed.
    office_location: [NPQS-KAT, SEA-CMB, AIR-BIA, AIR-JAF, AIR-CIAR, AIR-MRIA,
                      SEA-HAM, SEA-KKS, PO-CME, PO-GAL, PO-KDY, PO-JAF]

2. In one-trade-artifacts, update npqs/npqs_application_review/ (needs the refid schema PR merged first, since additionalProperties: false rejects the block until then):

npqs_application_review_v1.taskconfig.json — add:

"refid": {
  "issuer": "NPQS",
  "idType": "application_id",
  "path": "/reference_number",
  "params": { "officeCode": "/nppo_office_location" }
}

reviewerinput_jsonform.json — mark the existing field read-only, so the officer can't overwrite a generated number:

"reference_number": { "type": "string", "title": "NPQS Reference Number", "readOnly": true }

3. ./start-dev.sh --clean-run npqs

4. Create an NPQS application in the Trader Portal and submit it.

5. Log in as npqs_officer and open the application — NPQS Reference Number is pre-filled and read-only, e.g. NPQS/NPQS-KAT/20260904/000001.

To check the guarantees: a second application from the same office gives 000002, a different office starts at 000001, and re-injecting an existing task keeps its number.

Outcome of the test

Checklist

• My code follows the project's style guidelines
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

Related Issues

Closes #306

Additional Context

Two gaps left out of scope, both recorded on #306: the number isn't searchable in the applications list (List projects the JSONB columns out), and FinalizeReview stores whatever the client posts — so a read-only form control is convention, not enforcement.

Summary by CodeRabbit

• New Features

  • Added optional reference ID generation for tasks during initial submission.
  • Supports configurable issuers, formats, lists, date-based scopes, sequences, and padding.
  • Generated IDs are inserted into reviewer responses and preserved during re-submission.
  • Added validation for reference ID configuration and required data parameters.
  • Added persistent sequence tracking across supported database deployments.

• Documentation

  • Added configuration examples and task configuration guidance for reference IDs, including failure behavior and required review controls.